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The State’s SPP/APR includes a State Systemic Improvement Plan (SSIP) that meets the requirements set forth for this 
indicator. 


Baseline and Targets 


Baseline Data 


FFY | 2013 


Communication | 17.4% 
Arts 


Mathematics | 20.4% 


FFY 2013 —FFY 2018 Targets 
FFY 2014 2015 2016 2017 2018 


Communication 18.4% 19.4% 20.9% 22.4% 23.9% 
Arts 
Mathematics 21.4% 22.4% 23.9% 25.4% 26.9% 
Description of Measure 


Missouri’s State Identified Measurable Results (SIMR) for students with disabilities is focused on improving 
proficiency outcomes in the areas of reading/language arts and mathematics. Approximately 350 schools are 
voluntarily participating in a multi-year pilot project to validate the selected effective practices. This pilot 
initiative is called the Collaborative Work. Our SIMR is based on the outcomes for these 350 schools, and we 
will scale the process if this pilot process produces anticipated results. Missouri’s SIMR, therefore, is to 
increase the percent of students with disabilities in tested grades who will perform at proficiency levels on state 
assessments in reading/language arts and mathematics in the Collaborative Work pilot schools by 6.5 
percentage points by 2018. 


Targets: Description of Stakeholder Input 


Over the past several years, the Missouri Department of Elementary and Secondary Education (DESE), Office 
of Special Education has frequently presented information to Missouri stakeholder groups. While we have 
always held our primary stakeholder group to be the Special Education Advisory Panel (SEAP) and our 
secondary stakeholder group to be the Missouri Council of Administrators of Special Education (MO-CASE), 
we have presented the concept, purpose, and direction of the SSIP; data analysis; evidence-based strategies; 
and, a description of the Collaborative Work (Missouri’s name for the pilot implementation of the SSIP) to 
numerous other groups. These groups most notably have been the Missouri Planning Council, the Blind Task 
Force (BTF), the Regional Professional Development Center (RPDC) directors and specialists, school 
administrators (mostly superintendents), and area supervisors (DESE staff who are liaisons with schools across 
the state). Support for the work has been generally positive. Stakeholders especially appreciated the way we 
summarized the review of data into a table that included findings and observations. The variety of ways data 
were connected and separated (especially into elementary compared to secondary) challenged the beliefs of 
many. The end result was a new appreciation for in-depth data analysis and stronger support for improving 
outcomes for students with disabilities in the areas of language arts and mathematics. While the data review 
clearly demonstrated there are multiple areas in need of improvement, no one challenged the focus on reading 


and mathematics as being the most critical regardless of disability category or grade level. 


One of the biggest hurdles was, and to some extent continues to be, the concept of improving, not because the 
building was labeled good or bad or given some accountability designation, but improving just because we can 
do it and it is the right thing to do. Accountability is so ingrained in our way of doing business that it now 
competes with our general sense of responsibility to others. We continue to hear strong support of the 
improvement process being initiated under the title of Collaborative Work (CW). Our SEAP especially 
appreciated that we successfully took the Collaborative Work and embedded it as part of the key strategic 
process in the Department’s Top 10 by 20 Plan. This may be the first time special education did not feel like an 
add-on to someone else’s document. As a result of the positioning of the Collaborative Work, we are seeing a 
significant effort on the part of other Offices within the agency to connect and align with our work. We are also 
making a significant effort to align what we do to the other quality improvement work supported throughout the 
agency—this is perhaps the beginning of systems development. 


Because of timing, the SEAP (our primary stakeholder group) was the only stakeholder group with whom we 
discussed the target setting for the State-identified Measurable Result(s) (SIMR). They again saw that we were 
applying business rules to the process of developing targets for the SIMR consistent with target setting for the 
SPP/APR. The only differences in the process were using a smaller set of schools (the 350 plus CW pilot 
schools) and setting the targets based on the pilot school data, not statewide data. The SEAP agreed that the 
appropriate approach to target setting was followed. 


Data Analysis 

A description of how the State identified and analyzed key data, including data from SPP/APR indicators, 618 
data collections, and other available data as applicable, to: (1) select the State-identified Measurable Result(s) 
for Children with Disabilities, and (2) identify root causes contributing to low performance. The description 
must include information about how the data were disaggregated by multiple variables (e.g., LEA, region, 
race/ethnicity, gender, disability category, placement, etc.). As part of its data analysis, the State should also 
consider compliance data and whether those data present potential barriers to improvement. In addition, if the 
State identifies any concerns about the quality of the data, the description must include how the State will 
address these concerns. Finally, if additional data are needed, the description should include the methods and 
timelines to collect and analyze the additional data. 


Process of identifying, selecting, and analyzing existing data: To guide the development of the Missouri 
State Systemic Improvement Plan (SSIP), data from a wide number of sources were collected and organized for 
analysis to determine the SIMR and root causes contributing to low performance (see Table 1 and Charts 2 and 
3). These sources included trend data from the Part B SPP/APR Indicators, 618 data for the past five years, and 
special education compliance data from the past three years. These data were incorporated into a PowerPoint 
called “Special Education At A Glance” to aid in data discussions with various stakeholder groups. Data were 
reviewed over several months with both internal and external stakeholder groups. Internally, data were shared 
with special education staff at bi-weekly staff meetings and with other department staff in various ways (Office 
staff meetings, Executive Leadership Team meetings, DESE area supervisors, Department Data Team meetings, 
and Regional Professional Development Center directors and staff). Externally, a chart summarizing multiple 
data reviews was developed and presented to the Missouri Planning Council, Blind Task Force, Special 
Education Advisory Panel, and Missouri Council of Administrators of Special Education. Selected data was 
also shared at various meetings and conferences, such as Special Education Administrators Conference, Council 
for Exceptional Children, and the Commissioner’s Conference (primary audience is superintendents and central 
office staff). 


Data were organized by Office of Special Education Programs (OSEP) indicator with particular emphasis on: 
e Student performance data—Missouri Assessment Program (MAP) 


Discipline 

Attendance 

Transition 
Post-secondary outcomes 
Poverty 

Compliance 


Data were disaggregated in a number of ways and were looked at for the entire state, for the nine RPDC 
regions, and for individual districts. Some of the major ways in which data were grouped are as follows: 
e Data for all SWDs compared with all students 
e Data for 6-7 major SWD categories compared with all students, all students with disabilities, and with 
each other. Some data were separated into age/grade span groupings to get a better look at variations 
between elementary and secondary 
e Scatterplots were developed to show relationships among poverty/proficiency/inclusion and 
poverty/proficiency/incidence rate at the district level 
e SWD proficiency data were compared to non-disabled, minority, English language learners, and low- 
socio-economic data 
e All districts were scanned to see if there were any districts where SWD performed at a significantly 
higher level than would be predicted by socio-economic data 


Root Cause 

When looking at the possible root causes of low student performance/achievement gaps across an entire state, 
Missouri recognized that “The causes of low performance/achievement gaps are multiple and complexly 
interrelated, and they vary from school to school, district to district, and community to community.” 
(http://www.nea.org/). For this reason, the state felt it would be most effective and efficient to look at those 
factors that have been identified in various studies as contributing to low performance/achievement gaps and to 
structure an improvement framework which would allow individual districts and buildings to identify and 
address with evidence-based activities those areas which were having the greatest impact on low 
performance/achievement gaps in that district/building. Much of our information related to root cause analysis 
comes from years of accumulated reports generated by on-site reviews of districts and schools across the state. 
What was often found was the widespread use of instructional practices not grounded in research and the lack of 
systematic implementation of effective teaching and learning practices. What was observed in the on-site 
reviews as contributing causes of low performance were validated through John Hattie’s synthesis of over 800 
meta-analyses relating to achievement. Other contributing factors observed were the lack of collaborative 
cultures and infrequent use of data to guide improvement. These observations were also consistent with the 
Moving Your Numbers study guided by the National Center for Educational Outcomes. 


Data Quality 

Primary data quality concerns are in the area of ECSE. A main issue is the lack of a single early childhood 
assessment used across the state for both students with disabilities and non-disabled students. An assessment 
(the Desired Results Developmental Profile (DRDP)) has been adopted by the State Board of Education and is 
in the process of being implemented in state-sponsored Early Childhood Programs and Head Start programs 
across the state, including Early Childhood Special Education. 


Overview of SPP/APR data 
The following table (Table 1) shows Part B SPP data spanning 2005-2013. Of the 32 items (representing 20 
indicators) on this list, 18 (56.2%) are at or above the target for the academic year 2012-2013. 


Table 1 


Green = Met 
Missouri Annual Performance Report (APR) Summary Red = Not Met 
rane Yellow = No Movement 
Determination based on 2012-13 data: Meets Requirements 
Determination based on 2011-12 data: Meets Requirements 
Determination based on 2010-11 data: Meets Requirements 
Determination based on 2009-10 data: Meets Requirements 
Determination based on 2008-09 data: Meets Requirements 
Determination based on 2007-08 data: Meets Requirements 
Determination based on 2006-07 data: Meets Requirements 
Determination based on 2005-06 data: Needs Assistance 
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The following Data Summary Chart (Chart 1) provides an overview of the types of data, findings (factual), and 
observations (interpretations) from a review of essential data elements which helped inform the selection of the 
critical area of need to be addressed in the SSIP. The data is clustered in a variety of ways because it is 
important to understand variations among students with disabilities. It is also important to show how data from 
various sources are used. 


Chart 1: Data Summary Chart 


Data Topic Description Findings Observations 
Socio- Identified e MO has a 49% F/R rate e Almost 60% of SWDs are on 
Economic percentage of IEPs |e MOhasa disability rate of F/R lunch 
in the following 12.56% e Students on free/reduced lunch 
categories: e Less than 10% of non-F/R are almost 1.5 times as likely to 
Unreduced, have IEPs have an IEP as students not on 
reduced, and free | ¢ 13% of reduced lunch free or reduced 
lunch have IEPs 
Sources: 3, 5, 16, e About 16% of free lunch 
20, 23 have IEPs 
Graduation Provides data re: e ALL students: 2011 = e Between 2011 and 2013 the 4 
rates graduation rates for 81.3%, 2013 = 87.7% an year graduation rate for SWDs 


“ALL” students and increase of 6.4% increased slightly better than for 


SWDs from 2011- 
2013; compared 
results for primary 
categories of SWDs 
in 2011 and 2012 
Sources: 1, 2, 11, 
23, 28 


SWDs: 2011 = 68.6%, 
2013 = 76.3% an increase 
of 7.7% 

LD and OHI had best rates 
in 2012 at 75.7 and 77.5% 
respectively 

ED had the lowest rate 
55.5% in 2012 


ALL students 

LD, OHI, and ED must all 
continue to improve to close the 
gap due to the numbers of 
students represented 

SWD grad rates improve by 
about 5% when using a 5-year 
rate over a 4-year rate 


Accountability 


2008-2010 data re: 
percent proficient 
for Dist/Sch w/30 
or more accountable 
students by 
category in CA and 
math 

Sources: 4, 5, 12, 
13; 18,.22;.25527; 
29 


State totals (including MAP- 


A): 


CA —“ALL” went from 
60% proficient to 62% 
(545 dist/1,998 schools) 
CA—IEP went from 
36.9% to 37.8% (320 
dist/760 schools) 
Math—“ALL” from 
55.3% to 61.7% 
Math—IEP 35.9% to 
39.6% 


IEP were the lowest performing 
subpopulation in both CA and 
Math over these 3 years 
Around 59% of districts and 
38% of schools have sufficient 
IEP student N-size to be 
included in the accountability 
system 

No school met the proficiency 
standards without MAP-A 
scores being included 


Elementary vs_ | Changes of e The total numbers of The most severe cases of LD, 
Secondary numbers and SWDs is relatively stable ED, AU, and MR/DD are likely 
percentages in the in the two age groupings identified in the elementary 
most prominent e Number of secondary years with additional but less 
SWD categories S/Lng Impairment is 1/4" severe cases identified in the 
from elementary to of elementary #s secondary years 
secondary using age |e Numbers of LD and ED Speech impairment has a high 
groupings of 6-11 almost double in the incidence rate in the elementary 
and 12-17 secondary and a very low incidence rate in 
Sources: 3, 26 e Numbers of OHI and MR the secondary 
increase by 1.5 and 1.4 Speech and language 
respectively in the impairment occur less frequently 
secondary in secondary but likely represent 
most severe of each 
Proficiency Compares SWD e All CA 2007 = 44.7%, Over time, SWDs are increasing 
levels (excludes MAP-A) 2012 = 55% in the top 2 proficiency levels at 


to ALL students; 
compares major 
categories of SWDs 
to each other; 
compares major 
categories of SWDs 
by age groupings 
(6-11 and 12-17) 
Sources: 4, 5, 7, 
12, 13, 17, 18, 22, 
25, 27 


SWD CA 2007 = 17.6%, 
2012 = 27.4% 

All math 2007 = 45%, 
2012 = 55% 

SWD math 2007 = 20.9%, 
2012 = 29.8% 

Elem SWD CA 
proficiency = 18.8% 

Sec SWD CA proficiency 
= 29.4% 

Elem SWD math 
proficiency = 23.2% 

Sec SWD math 


rates similar to All students 
SWD have narrower gaps w/All 
in mathematics and science 
Proficiency scores for Speech 
Impairment are just slightly 
below those of All students 

LD, OHI, and ED need marked 
improvement to close any gap 
and primarily account for the 
significant gap between SWDs 
and All students 

No district could be identified as 
exemplary for IEP student 


proficiency = 23.7% 


proficiency performance 


Discipline Compares ED students are 3-4 Xs Discipline rates for SWDs is 
discipline data for more likely to be involved about twice as high as for non- 
each category of in a disciplinary issue disabled students 
SWD relative to the OHI are 1.5-1.8 Xs more ED and LD more likely to 
percentage of make- likely experience long term 
up of SWD LD are about as likely as disciplinary removal 
population SWDs 
Compares SWD to Students with autism are % 
non-disabled as likely as other SWDs to 
students be involved ina 
Sources: 3, 8, 23 disciplinary issue 

Sp/Lg Impaired are 4% to 4 
as likely 

LRE Compares LRE by MO is lower than Nat. avg MR has the lowest rates of 
SWD category and in > 80% or more but inclusion of the major categories 
by age spans 6-11 surpasses Nat. avg when Secondary appears less inclusive 
and 12-17 adding in > 40% but that is mostly a result of the 
Sources: 23, 26 85% of SWDs spend 40% change in numbers of Sp/Lng 

or more of their time in from the elementary 
general ed. classrooms 
SWD All districts placed Q1 (lowest poverty) had In 3 of 4 quartiles, achievement 


incidence rates 
and 
proficiency of 


in a quartile based 
on poverty (equal 
numbers in each). 


the highest rate of 
performance (31.9% of Q1 
districts were above 30% 


improved as incidence rates 
increased 
Q3 (57.4-67.6% poverty) 


districts MAP-A not proficiency) achievement was largely 
viewed by included 80.8% of all districts had unaffected by incidence rates 
poverty strata | Sources: 30 proficiency rates lower (Q3 had lowest rates of 

than 30% inclusion) 
SWD inclusion | All districts placed 36.6% of all districts had In low poverty districts (Q1 and 
and in a quartile based inclusion rates greater than Q2), achievement tends to 


proficiency of 
districts 
viewed by 
poverty strata 


on poverty (equal 
numbers in each). 
MAP-A not 
included in these 
counts. Inclusion = 
>79% of time in 
reg. ed. 

Sources: 30 


70% 

Q3 (57.4-67.6% poverty) 
had the lowest overall 
inclusion rates (74.6% of 
districts with less than 
70% inclusion) 


improve as inclusion increases 
Q3 largely unaffected by 
inclusion rates 

In high poverty districts (Q4), 
performance decreases as 
inclusion increases 


Post- 
Secondary 


Reviews post- 
secondary outcomes 
(college and career) 
Sources: 1, 2, 3, 7, 
16, 17, 23, 28 


MO data show little 
change over the past 
several years. 
Attendance at both 2-year 
colleges and non-college 
training is slightly higher 
for SWDs than for all 
students 

Attendance at 4-year 
college lags significantly 
for SWD (about 23% 


7 


LD and autism exhibit positive 
college attendance at rates not 
forecast by proficiency rates 
ED and TBI lag significantly in 
college enrollment and 
competitive employment 


points difference) 
e Competitive employment 

for SWD is 6-7% higher 
for SWD than for All 
students 

Statewide Reviews results of |e Many services focusedin |e No initiative seemed to be 

System of the SSOS the same districts and accelerating improvement 

Support Sources: 6, 19, 24 buildings regionally or statewide 

(SSOS) e Difficult to evaluate e Few initiatives had a scalable 
results of the SpEd business model 
improvement grants e Many districts/schools not 

e No region showed results getting improvement supports 
that could not have been even though results for 
predicted by subpopulations were uniformly 
demographics poor across the state 
Chart 2: Data Sources for Chart 1 
# Major Source Name Description 

1 2009-2011 Post-Sec Summary Post secondary data over 3 years 

2 09-10 Post-Sec Follow-up 2009-10 post secondary data by category 

3 2012-13 Misc SPED data Child count, discipline, and removal by category and offense, 
exiting, child complaints, due process and resolution, and 
assessment participation 

4 Accreditation 2009-11 (3-year) proficiency and growth data 

fs) Achievement Level 4 Report 2010-12 (3-year) MAP data by grade level and disability category 

6 Areas of Focus by School District | Grant focus areas by region and type 

7 CTE Student Counts 2011 participation of IEP students in Career Technical Education 
(CTE) by course types 

8 Discipline by Length 2012 and 2013 All/IEP/Non-IEP incidence data by offense, 
weapons, removal type, and length 

a Due Process States 2011-12 and 12-13 due process resolution 

10 __| ECSE total Numbers of ECSE and cost/child by district 

11 GHS-RMA(2) 1998-2011 graduated high school (GHS) and reached maximum 
age (RMA) 

12 Largest 25 Number not proficient in the largest 25 districts 

13. | MAP and F/R 2011-13 (3 years) MAP and F/R comparison 

14 | Mediations 2010-2012 Mediations and results for 2 years 

15 Missouri Census 2008 2008 Census Data 

16 | Outcomes data 2011-12 MAP, Child Complaints/Due Process, Discipline, ECO, 
ECSE, Grad/DO (by disability category) 

17 __| Post-Sec Compares All and SWD post-secondary outcomes 

18 Proficiency 0, 1, 2, and 2R 4 reports: CA grades 3-5 from 2005-06 through 2010-11 

19 RPDC Rankings Ranks RPDCs based on various SpecEd indicators 

20 Socio-Econ (13) Free/Reduced lunch counts for IEP/Non-IEP 

21 Special Education at a Glance 4 2 PowerPoints to lead data discussions 

Follow-up and (2) 

22 SPED HQT Core courses taught by SpecEd teachers 

23 SPP At a Glance 12-13 Multi-year APR data 

24 SSOS Buildings 2011 buildings with services—Priority, Focus, PBIS, PLC, etc. 


25 Proficiency 3-year comparison 3 years proficiency of sub populations with growth calculations 
for 2011 
26 State Child Count 2012 Child count of major categories with age spans 
27 Statewide by Disability 2009- CA and math grade level and top 2 levels of proficiency by 
2010-2011 disability category 
28 Statewide Follow-up by 2009-10 thru 2011-12 (3 years) post secondary outcomes by 
disability(2) disability category 
29 Subpopdata 2011 Disaggregated by subpopulation data and percent top 2 for 
CA and math 
30 Scatterplots Intersects poverty, proficiency, incidence, inclusion data 


Regional Data Review 


Data were also reviewed by region of the State. Missouri has nine Regional Professional Development Centers 
(RPDCs) which provide professional development and technical assistance to schools. They are often involved 
in implementing statewide initiatives. Because they are an important part of the system of support 
infrastructure, it was important to see if any of the RPDCs could be identified as more effective in providing 
training and technical assistance. The concept being that if one or more centers were doing better than the 
others some of their practices should be incorporated into the improvement process considerations. Data used 
for this review included statewide assessment, discipline and poverty data for all students and separately for 
students with disabilities, and data related to participation/non-participation in major initiatives such as School- 
Wide Positive Behavior Supports (SW-PBS) and Professional Learning Communities (PLC). This data review 
failed to identify one or more of the RPDCs as doing better than the others. Almost all differences in outcomes 
could be explained by poverty variations. Though no differences were identified, the review was included in 
determinations regarding the analysis of the statewide infrastructure as it indicated a need to improve the quality 
of assistance from the RPDCs. 


Summary 


The data analysis identified a number of areas needing attention and improvement including early childhood 
outcomes, discipline, graduation and dropout rates, and student academic performance in English language arts 
and mathematics. The fact that performance of SWDs was so far below state targets provided a strong incentive 
to focus on that area. The thought that student performance seemed like a driver of many other areas such as 
discipline, attendance, dropout rates, graduation rates, and post-school outcomes made it difficult to ignore. No 
stakeholder group or stakeholder group individual argued for a different area of focus. The initiative to focus 
the States Systemic Improvement Plan (SSIP) on improving student academic outcomes was named the 
Collaborative Work (CW). The CW work will be mentioned in the infrastructure analysis as it was important to 
analyze the Statewide System of Supports within the context of successfully implementing a major initiative 
focused on student performance outcomes for students with disabilities. 


Analysis of State Infrastructure to Support Improvement and Build Capacity 

A description of how the State analyzed the capacity of its current infrastructure to support improvement and 
build capacity in LEAs to implement, scale up, and sustain the use of evidence-based practices to improve 
results for children with disabilities. State systems that make up its infrastructure include, at a minimum: 
governance, fiscal, quality standards, professional development, data, technical assistance, and 
accountability/monitoring. The description must include current strengths of the systems, the extent the systems 
are coordinated, and areas for improvement of functioning within and across the systems. The State must also 
identify current State-level improvement plans and initiatives, including special and general education 
improvement plans and initiatives, and describe the extent that these initiatives are aligned, and how they are, or 


could be, integrated with the SSIP. Finally, the State should identify representatives (e.g., offices, agencies, 
positions, individuals, and other stakeholders) that were involved in developing Phase I of the SSIP and that 
will be involved in developing and implementing Phase II of the SSIP. 


Overview of Statewide System of Support 


The initial analysis of the statewide system of support focused on what it would take to successfully 
implement the Collaborative Work (CW) (Missouri’s pilot for the SSIP). By being very specific about the 
design of the CW, we could make better judgments regarding what it takes to conduct a successful pilot in 
schools across the state and then scale the process potentially to any school or district wanting to implement 
the process. By understanding the strengths and concerns of the present system, we could be intentional 
about developing resources and processes to address concerns and take advantage of strengths. The initial 
question was who/what was best situated to support districts and schools statewide and be able to sustain that 
support over many years. Understanding the time and uncertainty required in building a new statewide 
system through competitive contract and the real capacity limitations inside the agency much of the attention 
quickly focused on the Regional Professional Development Centers (RPDCs). These regional centers were 
one of the few existing resources that seemed viable to transform into a statewide support system. Several 
positive attributes were evident from the beginning: 


e The state has a significant investment in and reliance on the RPDCs to provide ongoing supports but not 
targeted interventions. 

e The RPDCs have been trained in some practices that are valued as part of the Statewide System of 
Support (SSOS) such as collaborative team decision making (SW-PBS and PLC) and data driven 
decision-making. 

e RPDCs represent one of the few organized resources available in all areas/regions of the state and are 
ideally situated to help districts/schools statewide. 


Designing the Collaborative Work helped identify important attributes of a statewide system that would be 
essential. The system attributes were viewed not on the ability of any single RPDC but on the perceived 
ability of the Centers at that point in time to function as a cohesive statewide unit. The following risk 
analysis chart represents a snapshot of our initial major areas of interest and our judgments of the health of 
the “system” based on past experience, observations, knowledge, and data. 


Table 2: Risk Analysis 


Category SSOS Attribute Low Medium | High 
Risk | Risk Risk 

Governance Independence related to each other and of the state X 
Fiscal Fiscal/business plans in place for sustaining/ Xx 

maintaining/expanding are observable 

Defined process to bring projects to scale Xx 
Quality Uniform quality standards for staff skills, knowledge, and Xx 
Standards competencies 

Uniform quality standards for training staff X 

Depth of knowledge related to curriculum X 

In-house expertise in areas specific to the Collaborative Xx 

Work 

Capability and capacity to provide individualized coaching X 

Capability and capacity to provide group coaching XxX 
Professional Uniform quality standards for PD delivery Xx 
Development 
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Uniform quality process for materials development X 


Strategic use of technology to enhance and bring efficiency Xx 
to technical assistance/professional development 
Data Capacity | Use of data to guide decisions observed in the RPDC XxX 
operations 
Technical Use of measurements to evaluate progress and success Xx 
Assistance and 
Accountability 


Implementation Driver Analysis 

The framework of implementation drivers, as articulated by the National Implementation Research Network 
(NIRN), is a helpful lens for pulling together and reflecting on the multiple components of the state system. 
The NIRN research identifies eight implementation drivers. Four of the drivers focus on competency 
(performance assessment/fidelity, selection, training, and coaching) and three focus on organizational systems 
(decision-support, data system, and facilitative administration). The remaining driver is leadership. 


Performance Assessment: How does Missouri know that improvement activities are implemented with 
fidelity and are leading to desired change? As mentioned, data to describe a cohesive implementation 
story is lacking. However, there are actively implemented components to build upon. For example, 
Missouri School-wide Positive Behavior Supports and Missouri Professional Learning Communities both 
incorporate implementation measures (SW-PBS School-wide Evaluation Tool (SET) and PLC Benchmark 
Assessment Tool (BAT)) and these measures are paired with walk-through/observation/interview to give 
a picture of school-wide implementation. Based on this data, participating schools receive a data report 
and coaching for improvement occurs. While these implementation measures have been very helpful in 
providing a data rich description of implementation, there are remaining data gaps. Expected fidelity of 
coaching has not been formalized thus is not consistent. The links between these data points and the 
intervention components has not been standardized. Additionally, these two initiatives are limited to 
social-emotional/behavior outcomes and school-wide collaborative culture. Performance assessment of 
the implementation of effective teaching/learning practices is in the beginning stages of development and 
initial implementation. 


There is a need to give LEAs a more cohesive data picture of how their teaching practices and school- 
wide systems are affecting student learning. This will require collaborative problem-solving to determine 
how current data is supportive, completion of all implementation measurement tools with explicit 
instructions for use in LEAs, and for use by RPDCs to better support educators, development of a LEA 
Profile/Dashboard to make tracking progress and interpreting data less cumbersome for schools, and 
continued commitment to providing LEAs and RPDCs with data in a timely manner. Through Missouri 
Collaborative Work progress is underway to address these gaps. 


Selection: What processes are in place to guide the selection of improvement activities, foci of 
professional development, and staff/consultants/contracts to conduct the work? In the spring of 2012, 
more than 120 stakeholders met in Jefferson City, Missouri, to review state data, provide diverse 
perspectives, and reflect on strengths and challenges of the new State System of Support (SOSS). During 
this meeting, 69 participants (64 representing the RPDCs and 5 DESE staff) completed a survey rating the 
level of need across Missouri in several areas. For future state professional development, participants 
rated topics from ‘Very Important (5)’ to ‘Not Important (1)’ as potential focus areas of need in Missouri 
(see Table below). Respondents averaged 5.39 years of experience working in a Missouri RPDC, and the 
majority of respondents specialized in Professional Learning Communities (PLC), special education, and 
School-wide Positive Behavioral Supports (SW-PBS). 
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Table 3: Needs Assessment 
Areas of Need (n=68) 


Very Not 
Areas of Need Important Important 
School culture 53 0 
Collaborative data teams 49 
The use of formative assessments 56 
Alignment to the Missouri Core Academic Standards 
Teaching and learning practices 57 


Later, a DESE leadership group formed to assess needs across Missouri. This interdisciplinary team of 
ten DESE staff represented three DESE offices. The team reviewed SPP data and recognized the 
importance of involving additional critical stakeholders to the process in order to accurately identify areas 
of need throughout Missouri. A key result of this stakeholder meeting was a prioritized state needs plan. 
While other topics arose, the stakeholder group agreed that addressing teaching and learning practices and 
the use of formative assessment are critical for better preparing students with disabilities for learning and 
life. They agreed that there remain significant needs across the state, identified as: (a) lack of 
achievement in Communication Arts, (b) lack of achievement in Math, and (c) lack of access to the 
general education environment to be addressed by the Collaborative Work initiative. 


To launch the initiative, elementary and middle schools (N=378) were invited to participate. The student 
demographics (Table 3) of 359 participating Collaborative Work schools in 2012-2013 were consistent 
with the representative demographics statewide. 


Table 4: Comparison of Demographics 
Comparison of Student Enrollment Demographics: Collaborative Work Schools and Statewide 
(2012-2013; n=359) 
Collaborative Work Schools — Statewide 
Demographics (%) (%) 
Free/ Reduced Lunch 
>60% 39.3 41.6 
40-60% ppl 33.5 
<40% 25.6 25.0 
Urbanicity 
Rural 42.6 39.2 
Town 23.4 17.3 
City oy 15.0 
Suburb 24.2 28.5 
Ethnicity 
White 79.4 76.2 
Hispanic 3.0 4.6 
Black 10.6 15.4 
Asian 1.5 1.4 


For these schools, supports for building collaborative data teams, use of effective teaching and learning 
practices, and common formative assessments are provided with a goal of reaching school-wide 
implementation with fidelity. The DESE relies on contracts with RPDCs to supplement the State’s 
capacity to meet the needs of LEAs. Contracts with the RPDCs provide for regional and on-site 
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professional development for educators. Through the State Personnel Development Grant, DESE has 
contracted with University of Missouri-Kansas City and the University of Kansas to support project 
management, development of professional development content, development of implementation 
measures, and comprehensive evaluation. With all of these contracts, there are scopes of work outlining 
expectations and deliverables. Additionally, within the contract for the RPDCs are job descriptions 
outlining expected expertise of RPDC personnel working directly with LEAs. 


Training & Technical Assistance: Who are the partners in delivering training and technical 
assistance, how is quality assured, and how is the process driven by data? The RPDCs are the primary 
providers of training and technical assistance and support for development of content and measurement of 
implementation occurs in partnership with institutions of higher education (IHEs). The recommended 
“dosage” of training and technical assistance necessary for support implementation progress has not been 
determined; however, initial data is currently being reviewed to inform an initial discussion about how 
intensity should be differentiated based on LEA/educator/student needs. This data is collected through 
RPDC activity logs in which RPDC personnel log their interactions with LEAs and report the nature and 
extent of their interactions. [See Quality Standards and Professional Development section below for a 
description of current practices. ] 


Through Collaborative Work, 20% of training is observed using a fidelity of training checklist developed 
for the project. Additionally, this checklist is used when the DESE hosts/provides professional 
development to the RPDC personnel. The tool represents a compilation of research-identified indicators 
that should be present in high quality professional development. Professional development training with a 
maximum of one item missed per domain on the checklist can be considered high quality. State 
Implementation Specialists use this checklist when observing 20% of the Collaborative Work training 
events. Data collected using this checklist from October 1, 2013, through November 30, 2014, shows 
95% of the observed trainings met criteria. The contents of the checklist address the following areas. 


PREPARATION 
1. Provides a description of the training with learning objectives prior to training 
2. Provides readings, activities, and/or questions to think about prior to the training 
3. Provides an agenda (i.e., schedule of topics to be presented and times) before or at the beginning of 
the training 
4. Quickly establishes or builds on previously established rapport with participants 
INTRODUCTION 
5. Connects the topic to participants’ context (e.g., community, school, district) 
6. Includes the empirical research foundation of the content (e.g., citations, verbal references to 
research literature, key researchers) 
7. Content builds on or relates to participants’ previous professional development 
8. Aligns with school/district/state/federal standards or goals 
9. Emphasizes impact of content on student learning outcomes 
DEMONSTRATION 
10. Builds shared vocabulary required to implement and sustain the practice 
11. Provides examples of the content/practice in use (e.g., case study, vignette) 
12. Illustrates the applicability of the material, knowledge, or practice to the participants’ context 
ENGAGEMENT 
13. Includes opportunities for participants to practice and/or rehearse new skills 
14. Includes opportunities for participants to express personal perspectives (e.g., experiences, thoughts 
on concept) 
15. Includes opportunities for participants to interact with each other related to training content 
16. Adheres to agenda and time constraints 
EVALUATION 
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17. Includes opportunities for participants to reflect on learning 

18. Includes discussion of specific indicators—related to the knowledge, material, or skills provided 
by the training—that would indicate a successful transfer to practice 

19. Engages participants in assessment of their acquisition of knowledge and skills 


Recently, measurement of content fidelity has been added to the checklist for fidelity of delivery of 
professional development. It is too early to draw conclusions from the content fidelity data. 


Coaching: How is coaching provided to support skill development? Coaching is provided through the 
Collaborative Work on specific teaching/learning practices, as well as through SW-PBS and MO PLC. 
Recently, criteria for fidelity of coaching have been established for Collaborative Work. State 
Implementation Specialists observe 20% of coaching provided to LEAs through Collaborative Work. 

The coaching checklist was formalized into an online tool beginning with summer 2014. Data reported on 
coaching event observed between July 2014 and December 2014 show nearly all coaching events met 
criteria (99%). The fidelity of coaching checklist addresses the following key components of coaching. 


PREPARATION 
1. Clarifies purpose and scope of the coaching session. 
2. Builds and maintains rapport, collegiality, and confidentiality with participants. 
FEEDBACK & SOLUTION DIALOGUE 
3. Facilitates conversation about what has gone well and where more support is needed. 
4. Facilitates conversation about relevant student data. 
5. Responds to ideas for improvement by validating and/or adding suggestions for changes in 
practice. 
6. Provides rationales for why changes are important and how changes will improve outcomes. 
7. Provides opportunity for reflection and clarification of recommendations. 
8. Supports suggestions for change in practice with examples of the content/practice in use. 
9. Offers opportunity or resources for guided practice. 
10. Facilitates identifying next steps from the coaching session. 
STRUCTURE 
11. Paces the conversation to allow time for questioning and processing of information. 
12. Adheres to established plan of coaching (e.g., frequency, schedule, and duration). 


The following graphic illustrates how training and coaching is supported across implementation levels. 


High quality professional development 
content, materials, and structures 


State 
Education 
Training and Agency 


coaching Regional 
Professional | 


Development |shared learning 


Classroom/ 
Building | Fidelity of delivery 
and content 
Educators 


Increased 
student | Fidelity of implementation 


learning 
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Decision Support Data System: To what extent does currently collected data address essential 
questions for improving student outcomes? Throughout this plan are multiple references to the scope of 
data collected, methods of sharing data with LEAs and schools, use of data to inform state and regional 
level decisions, and gaps in data collection and sharing methods that need to be addressed. Reliance on 
data reporting SPP/APR indicators is not sufficient for school improvement. Indicator data can point to 
shortcomings and/or show progress trends. However, explaining why the shortcomings or progress has 
occurred is based mostly on observational data from multiple site visits of Missouri School Improvement 
Teams over many years. These data show a lack of attention to the practices of teaching and learning. 
While declaring a lack of focus on teaching and learning as primary causes of poor performance, site visit 
data coupled with John Hattie’s synthesis of over 800 meta-analyses relating to achievement form a 
compelling argument. 


Facilitative Administration: How do collaborative processes ensure the development and 
implementation of policies and practice that support and reduce barriers to implementation? 
Upholding a commitment to implementing evidence-based practices often requires shifts in how and 
which education initiatives are adopted and supported. Currently, work is underway to more fully 
integrate School-wide Positive Behavior Supports and Professional Learning Communities practices to 
result in a statewide multi-tiered system of support (MTSS) model. A statewide MTSS framework is 
important to matching the teaching practices with the learning needs. While the MTSS work is focusing 
on the systems-level, the Collaborative Work is focusing on the classroom level of instruction. Oversight 
of the Collaborative Work (CW) is led by a state CW management team that meets monthly. Teams 
working on each have overlapping membership. Similarly, there is involvement from teams working on 
educator evaluation and teacher/leader standards. With a commitment to data-driven decisions, each team 
is committed to have a data-focused representative. In the recently adopted revised state strategic plan 
(Top 10 by 20 plan), implementation of effective teaching/learning practices (as developed through 
Collaborative Work) and leadership for supporting collaborative culture and data-driven decision-making, 
as well as parent involvement are clearly outlined. The revisions to the state strategic plan provide a 
strong state level endorsement for statewide implementation of the CW. 


Systems Interventions: How are issues of system barriers and potential solutions identified? The 
response to this question reiterates the importance of data at state, regional, and local levels describing 
implementation. Currently, data such as RPDC activity logs, anecdotal stories, and school compliance 
data collected through Collaborative Work suggest implementation gaps across levels. However, because 
the data systems are still being revised, it is difficult to determine with certainty where the gaps are and 
identify potential solutions. With the drive to meet the deadlines specified in the Top 10 by 20 plan, there 
is high interest in reviewing all current and planned data tools to determine how they might be better 
integrated and used to identify system issues and promote solutions. 


Leadership: Do leadership have the knowledge, skills, and authority to support school improvement? 
Effective leadership at all levels (state, regional, local) is critical. There are numerous opportunities for 
educator-leaders to receive professional development. Missouri Leadership for Excellence, Achievement, 
and Development (MoLEAD) combines online and face-to-face training with hands-on experiences and 
mentoring opportunities to enhance best practices in schools. More than 400 superintendents, principals, 
assistant principals, and teachers have already participated in the first two rounds of training. The DESE 
Office of Special Education provides training to new directors of special education. RPDCs provide 
leadership support to building and district administrators. The state agency’s Top 10 by 20 plan addresses 
leadership and the expected deliverable is a learning package focused on effective leadership for building- 
wide systems as well as supporting effective teaching/learning practices at the classroom level. To date, 
there is not specific data gathered on the fluency of leaders to support school-wide systems and 
classroom-level instruction. 
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Governance: The Department issues annual contracts to the RPDCs to carry-out state approved professional 
development to LEAs. Eight of the nine RPDCs are managed within a university structure. The remaining 
RPDC is managed within St. Louis Cooperating Schools Districts (now called EducationPlus), which is a 
consortium of school districts in St. Louis County and the surrounding area. 


Fiscal: Historically, RPDCs were reliant on flow-through state funds, school district professional 
development funds, and a few state contract funds. Funding appropriated to support the RPDCs dropped off 
severely several years ago. With the reduction in state funds, the RPDC business model changed to seek 
additional contracts to supplement state general revenue funding. The Department issues annual contracts to 
RPDCs to cover initiative-focused professional development to LEAs. The contracted amount varies by 
RPDC based on the number of participating LEAs in the initiative for which professional development is to 
be provided. 


Quality Standards: There is limited history of uniform quality standards or consistency of practice and 
training across the state. Statewide initiatives such as School-wide Positive Behavior Support (SW-PBS) and 
Professional Learning Communities (PLC) have developed protocols and expectations for implementation 
fidelity and scaling-up. With these two initiatives, there has been improved consistency of behavior 
interventions and collaborative teaming. With the Collaborative Work initiative that began development in 
2011, achieving statewide consistency while balancing regional flexibility to address the diversity of needs 
across regions and LEAs has been a goal. To improve statewide consistency, the CW modules were 
developed with personnel trained in SW-PBS and PLC. The process was intended to reduce inconsistencies 
between these two initiatives and to build CW modules reflecting the similarities of the two initiatives. 
Through Collaborative Work, criteria for uniform quality standards and consistency of practice has been 
established at the level of delivery of professional development to LEAs (regional) and at the level school- 
wide implementation (local). Specific activities for monitoring the quality of implementation are as follows. 

e Shared Learning events provide consistency of initial and continuing training and coaching to the 
RPDC network. 

e High Quality Professional Development Standards provide criteria for quality delivery of training and 
coaching to LEAs. 

e Structured Learning Packages include professional development content to be delivered to LEAs and 
incorporate research-based adult learning strategies. 

e State Implementation Specialists observe the delivery of professional development, record 
observation of quality on the project-developed checklist, and provide follow-up coaching to the 
professional development provider with recommendations for improving the delivery of content and 
interactions with educator-learners. 

e Practice Profiles accompany each Learning Package and are intended to guide implementation and 
observation of teaching and learning practices in the classroom. 

e Fidelity checklists also accompany each Learning Package and are intended as a guide for educators 
working to improve implementation of the practice in the classroom. 

e Working model of school-wide full implementation. 


Professional Development: The RPDC network is the primary source of professional development for 
LEAs. Through the Collaborative Work initiative, funded by the State Personnel Development Grant, there 
has been increased attention on consistency of quality of professional development across regions. A 
process for developing, vetting, rolling-out content for RPDC delivery, and expectations for how participating 
schools engage with the RPDC in adopting the practice and improving implementation has been clearly 
outlined. 

1. Department determines focus of learning package. 

2. A team of in-state experts, who work at one or more of the RPDCs, are recruited to develop the 

learning package. 
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3. Draft learning package is peer reviewed and feedback to the development team is provided. 

4. Learning package is revised. 

5. Learning package is prepped for vetting (formatting, copyright check, etc.) 

6. Learning package is vetted by a team comprised of RPDC professional development providers. 
7. Vetting team provides feedback and learning package is revised again as needed. 

8. Learning package is prepped for roll-out (re-check after latest revisions). 

9. Learning package is presented (rolled-out) by the development team to the RPDC network. 

10. Learning package materials are posted to project website http://moedu-sail.org/. 


Data Capacity: Being able to engage in data-driven decision-making is valued at all levels of 
implementation (state, regional, and local). Most of the RPDC personnel received data team training through 
The Leadership and Learning Center. That training continues to influence their regional approach to data- 
driven conversations and has influenced the data-based decision-making learning package delivered to LEAs. 
However, despite this progress, substantial gaps remain. At both the regional and local levels, there is a 
tendency to feel overwhelmed by data-overload. Teams struggle to match a dilemma or question with the 
best-suited data point(s). The analysis indicated that while the RPDCs were highly skilled in training on 
data, they lacked real skills in using data to monitor their progress and the progress of their client LEAs. Asa 
starting point, the Collaborative Work schools are required to move through a common formative assessment 
approach, submit their data, and the RPDC provides them feedback. By making this process required and 
transparent between the LEA and the RPDC, there is opportunity for guiding educators through an active 
data-driven process as well as an opportunity for the RPDCs to collectively consider their regional data and 
shape their professional development accordingly. 


TA and Accountability: Progress rubrics were developed as a mechanism for LEAs to monitor adherence to 
Collaborative Work expectations and for RPDCs and the State to monitor regional and school progress. 
These rubrics were recently launched; therefore, data has not yet been collected. 


The components of the building-level rubric include the following. 
COLLABORATIVE DATA TEAMS (CDTs) 
1. All certified instructional staffs are included on regular education CDTs. 
2. All CDTs meet regularly, frequently, and consistently. 
3. CDTs use appropriate protocols and processes to ensure productive meetings. 
INSTRUCTIONAL/LEARNING PRACTICES 
4. Each building selects 2 instructional practices to master during the year. 
5. All instructional staffs are trained in the instructional practice and implement with fidelity. 
6. An effective practice is used and a CFA is administered. Based on the results, students who have 
not met proficiency standards are re-taught and then re-tested for mastery of the content. 
7. Each grade level develops and administers a CFA during at least 5 cycles throughout the school 
year. 
PARTICIPATION OF SPECIAL EDUCATORS 
8. Special education teachers are active, participating members of the regular grade-level and 
content-specific CDTs. 
TEACHER MASTERY 
9. Teachers of students with disabilities, both those certified as special education and regular 
education teachers who have students with disabilities in their classrooms, demonstrate mastery of 
selected effective practices. 


The components of the RPDC-level rubric include the following. 
TRAINING AND DISSEMINATION 
1. Only staffs trained in the use of DESE approved Collaborative Work process, materials, and 
practice provide training. 
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2. Learning packages are delivered with fidelity. 
COLLABORATION 

3. Centers use collaborative data teams to make decisions about the effectiveness of implementation 

of the CW at both the building and regional center levels. 

COACHING 

4. Professional development is observed to ensure high quality and continued improvement. 
ACCOUNTABILITY 

5. CW buildings implement the expectations of the CW initiative. 

6. Centers collect and submit CFA forms. 

7. Staffs contracted for CW work meet the DESE expectations for serving CW buildings. 


Summary 


During the infrastructure analysis, Missouri identified strengths and weaknesses in developing a statewide 
system of support that would help build and maintain the Collaborative Work and other large scale statewide 
initiatives. The analysis results proved useful in helping guide intentional decisions about how to reduce, 
mitigate, or eliminate weaknesses and how to take advantage of strengths in the support structure. 


State-identified Measurable Result(s) for Children with Disabilities 


A statement of the result(s) the State intends to achieve through the implementation of the SSIP. 


To increase the percent of students with disabilities in tested grades who will perform at proficiency levels on 
state assessments in reading/language arts and mathematics in the Collaborative Work pilot schools by 6.5 
percentage points by 2018. 


A description of the result(s) the State intends to achieve through the implementation of the SSIP. The State- 
identified result(s) must be aligned to an SPP/APR indicator or a component of an SPP/APR indicator. The 
State-identified result(s) must be clearly based on the Data and State Infrastructure Analyses and must be a 
child-level outcome in contrast to a process outcome. The State may select a single result (e.g., increasing the 
graduation rate for children with disabilities) or a cluster of related results (e.g., increasing the graduation rate 
and decreasing the dropout rate for children with disabilities). 


Academic success is a primary goal and is closely linked to other indicators such as discipline, attendance, 
drop-out rate, graduation rate, and post-secondary outcomes. Proficiency in reading/language arts and 
mathematics is a gateway to other academic success and can influence discipline, attendance, and 
graduation/drop-out rates. Since 85% of students with disabilities are participating in the regular program of 
instruction 40% or more of the time, the decision is to focus on improving academic outcomes in the general 
education setting. The measure will be an increase in proficiency in reading and mathematics for students with 
disabilities on the state assessments. Initial years of the pilot will measure the potential of the Collaborative 
Work initiative to accelerate improved academic outcomes for students with disabilities in all state tested grades 
at accelerated rates in CW schools. 


It was clear from the data analysis that Missouri had several areas of improvement from which to choose. All 
are important, but at the end of the day, the agency and its primary stakeholders agreed that providing additional 
supports to improve the teaching/learning process in the regular classroom was the primary need and should not 
be delayed. In the area of student performance outcomes, the data showed significant gaps between all students 
and students with disabilities in high performing schools and low performance of all students, including 
students with disabilities, in low performing schools. Data also showed that the percentage of time spent by 
most students with disabilities in the regular classroom opened the door for improving outcomes for students 
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with disabilities by improving learning in the general education classroom where most academic instruction 
occurs. Root cause analysis of years of on-site reviews indicated little to no focus on the teaching and learning 
process. This suggests that most teachers use the same teaching practices they learned at the beginning of their 
careers. The areas of reading/language arts and mathematics were chosen for measuring progress since valid 
and reliable statewide assessments are available in those content areas. 


Reaching the academic performance targets requires a strong supportive infrastructure. Missouri reviewed the 
existing infrastructure (articulated in a previous section) of the state related to providing ongoing training and 
supports. Strengths and weaknesses were identified as they related to accomplishing the desired work. 
Strengths of the Regional Professional Development Centers (RPDCs) included training/credentialing in using 
data to drive decisions and training in collaborative teaming from various models. Major weaknesses were the 
lack of consistency, cohesion, and adherence to quality standards in many areas (See Table 2). It was clear that 
the RPDCs needed additional training and resources. Discussions with all of their staffs indicated that most 
were hesitant but willing to move from their comfort zones to try to address the need for improvement within 
the general classroom setting. However, most felt they were up to the task with additional help and time. 
Training, resources, tools, data, and quality control measures were put in place to assist the centers and their 
personnel to accomplish the work. Data are constantly reviewed to help guide the need for additional resources 
and training either regionally or statewide. 


SIMR Participant Selection 


In deciding which buildings should be invited to participate, we uniformly eliminated those schools and districts 
which were already receiving a variety of services due to state and federal accountability determinations. Those 
struggling districts and schools were already being supported and additional resources were probably going to 
be overwhelmed. Struggling districts and schools were primarily in two metropolitan parts of the state. 
Continuing to focus all efforts in the same two regions did not seem the appropriate process if we were looking 
to move the pilot to statewide implementation. Our best estimates were that it would take about four years to 
change the culture in many schools. Waiting for that time to lapse before building supports for all areas of the 
state was just not acceptable to our stakeholders who represented many other regions. So the first decision was 
to try to get participants from all nine (9) regions of the state. 


A second idea was to look for schools with significant gaps in academic improvement between the category of 
all students and students with disabilities. The data review required for that showed that the majority of gaps 
between these two categories existed mostly in the higher performing schools. So while performance gaps 
existed in those buildings, students with disabilities tended to perform above the statewide averages for students 
with disabilities. This was true in all assessed content areas. In many of the districts and schools where there 
were small or non-existent gaps, no student population was achieving at acceptable levels. The data review 
could also not identify any building where students with disabilities were performing at or near the level of all 
students in the building for two or more consecutive years. Those data indicated that all schools, whether high 
performing, low performing, or somewhere in between, needed help when it came to the academic performance 
of students with disabilities. 


The strategy then became to request volunteer schools in each region of the state with the only limitation as 
having enough students with disabilities to track progress. That limitation was waived to a slight degree in two 
very rural regions with small student populations. The decision to open the door to all schools had the side 
benefit of “focusing on improvement because that is our job” rather than “focusing on improvement because the 
State gave the school a label related to underperformance.” Stakeholders also saw this selection process to be 
an improvement and more appropriate to the mission of a state agency to help all schools. 


Fortunately, the demographics of the volunteer schools (explained earlier, see Table 4) showed a remarkable 
parallel with statewide demographics. This provides an extra degree of confidence that limited changes will be 
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necessary to scale the initiative. Other small considerations were a focus primarily on elementary buildings 
because of the similarity of the typical structure of elementary education and the transferability of the 
teaching/learning practices to all content areas. Encouraging use of the effective practices in all content areas 
will demonstrate that the effective practice is indeed embedded. Though we initially wanted to keep this at the 
elementary level, we did allow a few secondary schools into the mix to help us better understand how the 
process would need to change as we moved into schools structured around content areas. 


And finally, we determined that all teachers in the elementary buildings needed to be involved in the 
collaborative data teams. This was a significant departure from current initiatives such as SW-PBS and PLCs 
where schools could determine the number and type of teachers participating. Systems development and 
observations strongly indicate the need for whole school involvement as a requirement for changing and 
maintaining a collaborative culture as defined in the CW. 


Selection of Coherent Improvement Strategies 


An explanation of how the improvement strategies were selected, and why they are sound, logical, and aligned, 
and will lead to a measurable improvement in the State-identified result(s). The improvement strategies should 
include the strategies, identified through the Data and State Infrastructure Analyses, which are needed to 
improve the State infrastructure and to support LEA implementation of evidence-based practices to improve the 
State-identified Measurable Result(s) for Children with Disabilities. The State must describe how 
implementation of the improvement strategies will address identified root causes for low performance and 
ultimately build LEA capacity to achieve the State-identified Measurable Result(s) for Children with 
Disabilities. 


Overview 


The Missouri Collaborative Work is an educational framework which emphasizes the use of collaborative teams 
and data based decision-making to support effective teaching and learning practices at the classroom level with 
the goal of improved outcomes for all students, especially students with disabilities. 


Missouri Collaborative Work is focused on Visible Teaching and Learning. This work is informed by the 
research synthesis conducted by Dr. John Hattie and his continued work to create visible learning schools. This 
work is also informed by the Moving Your Numbers study conducted under the guidance of the National Center 
for Educational Outcomes. These studies articulate the need for focusing on a few things using a systems 
approach. The work clarified the need for alignment of state, regional, district, school, and classroom efforts. 
Additional guidance was provided through the continued work of educational, implementation science, 
professional development, and coaching leaders (i.e. Robert Marzano, the National Implementation Research 
Network, Jim Knight, Richard Dufour, and others). The primary message of Dr. Hattie’s work is “Know Thy 
Impact.” Making teaching and learning visible in Missouri is building school-wide models in which teachers 
and students maintain a teacher/learmer relationship characterized by the following: 
e Teachers are passionate about teaching and learning and their passion is contagious with students. 
e Teachers set learning intentions and success criteria aligned to Missouri Teaching/Learning Standards. 
e Teachers use effective instructional practices, conduct frequent checks for understanding, and provide 
specific feedback. 
e Students are taught how the learning intentions and success criteria are relevant and applicable, to 
articulate the extent to which learning has occurred, and identify needs for additional practice. 


Key teaching and learning practices, coupled with common formative assessments (CFA), analysis of results, 
and re-teaching can accelerate the learning of all students—even those presenting learning challenges. Finally, 
in Visible Learning for Teachers: Maximizing Impact on Learning, Dr. Hattie presents eight “mind frames” or 
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ways of thinking that together should underpin every action and decision in schools and systems. Each of these 
mind frames contributes to our understanding of how their implementation can facilitate effective learning if we 
integrate them into our practice. 
e Teachers/leaders believe that their fundamental task is to evaluate the effect of their teaching on 
students' learning and achievement. 
e Teachers/leaders believe that success and failure in student learning is about what they, as teachers or 
leaders, did or did not do... We are change agents! 
e Teachers/leaders want to talk more about the learning than the teaching. 
Teachers/leaders see assessment as feedback about their impact. 
Teachers/leaders engage in dialogue not monologue. 
Teachers/leaders enjoy the challenge and never retreat to "doing their best." 
Teachers/leaders believe that it is their role to develop positive relationships in classrooms/staffroom. 
Teachers/leaders inform all about the language of learning. 


Implementation 
Participating schools commit to carrying-out the following activities, steps, and practices. 


Commitment to Implementation Integrity 
e Selection, mastery, and implementation of a variety of effective instructional practices which have been 
proven to have a high effect on student outcomes. 
e Development and administration of common formative assessments by grade-level and aligned to the 
Missouri Learning Standards of mathematics/English Language Arts at least five (5) times annually. 
e Efficient and effective Collaborative Data teams at the building level using classroom data to make 
instructional decisions. 
e Monthly reports of data analysis: 
oO Practice used 
o Number of students assessed 
oO Number/% of students and SWD in level of proficient, close to proficient, far to go (likely to 
become proficient), and intervention students (not likely to become proficient) 
oO Re-teaching practice 
O Re-test results 


Core Practices 
e Collaborative data teams agree to use at least two effective teaching/learning practices they have 
selected to learn and use throughout the year. 
e The teams agree to teach to a specific Missouri Learning Standard in reading or mathematics using the 
selected effective practice. 
e The teams develop common formative assessments which they will use to determine the effectiveness of 
the teaching/learning practice and student progress. 

oO The teams analyze the data from the assessment and group students (all students, IEP students) 
into four performance levels which are the same as those used in the data. 

Oo Teams process proficient, close to proficient, far to go (likely to become proficient), and 
intervention students (not likely to become proficient). 

O The teams, based on the common formative assessment results, agree to a different 
teaching/learning practice to re-teach the students who are identified as far to go (likely to 
become proficient), and intervention students (not likely to become proficient). 

e Students are re-tested and the results are analyzed by the team. 


Implementation Expectations 
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e All work will be aligned with the Missouri Learning Standards and Teacher/Leader Standards. 
e Building administrator will: 

oO Assure that the participation expectations and agreements have been shared with all instructional 
staff. 

oO Assure that all instructional staff will be trained and participate on building collaborative data 
teams, provide support to instructional staffs who have expertise in an effective teaching practice 
to coach, and mentor colleagues. 

O Support and oversee the collaborative team process. 

oO Assure new staff will be trained/mentored on the collaborative work. 

e All teachers (including Special Education and special subjects [music, art, physical education, etc.]) will 
actively participate on a collaborative teacher team. 

oO Small buildings may only have one team covering all grade levels. 

oO Larger buildings may have 2 or more teams—some could have one per grade level. 

e Each building will: 

oO Identify a content area of English Language Arts or mathematics to focus their attention and to 
report progress. 

oO Select the “effective” teaching/learning practices for the year that all teachers will agree to use as 
part of the teaching/learning process. 

e Each building level collaborative data team will: 

oO develop, administer, score, and analyze results of grade appropriate common formative 
assessments aligned to a core academic standard. 

e A summary analysis based on the formative assessment will be shared with the RPDC consultant at the 
time the building submits the formative assessment. Basic information of the summary analysis will be: 

O Missouri Learning Standard addressed. 

oO Teaching/learning practice used. 

o Number and percent of students assessed in the grade-level. 

Oo Number and percent of all students (including students with IEPs) in each performance level on 

the assessment based on the initial administration. 

oO Number and percent of students with IEPs only in each performance level on the assessment 
based on the initial administration. 

o Teaching/learning practice used to re-teach students in the far to go (likely to become proficient), 
and Intervention students (not likely to become proficient). 

oO Number and percent of all students and students with IEPs in each performance level based on a 
re-test. 


Inclusion of students with IEPs in CFAs 

It is expected that most students with IEPs will participate in the grade-level CFAs just as they do other 
classroom assessments, either without or with accommodations indicated on their IEP. If a student is receiving 
all or most of their instruction in the general education classroom, then they should take the CFA. In the case of 
students with IEPs who are significantly below grade level and due to this, receive most or all of their 
instruction in a content area from their special education teacher and are identified as qualifying for the state 
level alternate assessment (MAP-A), these students may not be included in the classroom CFA. We would 
expect that the number of students with IEPs excluded from taking the CFAs would be very small. This 
initiative is not about accountability, but is intended to assist teachers to better understand and implement 
effective instructional practices and to improve the performance of all students, but especially students with 
IEPs. Research has shown that, under most circumstances, students with IEPs who are included in the general 
education classroom and curriculum achieve at higher levels than those who are not. 


Reporting requirements 
It is expected that each grade level in each building will report and share at least five (5) administrations of a 
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CFA each year. Two documents help validate these administrations. One document provides the CFA along 
with the scoring guide and links the assessment to a Missouri Learning Standard. The second document 
provides a summary of the assessment results and the reassessment results for all students and for students with 
disabilities. 


Sustaining the project 

Research is clear that 100% implementation across the building and preferably across the district, is required to 
get the types of improvements needed across the state. Buildings failing to fulfill their commitment to the 
project will be removed from participation. These buildings will not be replaced. The Office of Special 
Education (OSE) will support buildings to offset the costs of substitutes and stipends for training for at least 
three years. 


Implementation timeline 
Awareness and Recruitment 
Hold an Awareness and Recruitment Meeting with potential schools. This meeting can occur for individual 
schools or in regional cohorts. 
During the meeting: 
e Use the Missouri Collaborative Work Overview Learning Package to introduce the CW [Estimated time 
= 45 minutes] 
e Review the CW Common Understandings document 


Before proceeding, review and secure Statements of Commitments from buildings desiring to participate. 


Active Buildings 
Year 1 Buildings 
1. Orientation, implementation assessment, and planning 
e Use the Missouri Collaborative Work Overview Learning Package to orient all staff in the building to 
the CW [Estimated time = 45 minutes] 
e Use the CW Getting Started Guide and Practice Profiles to assess CW building’s level of 
implementation. 
oO Assist building in selecting Teaching Practice(s) to focus on for the year 
O Based upon results of the assessment and selection of effective teaching practice, plan CW 
professional development for the year with appropriate building staff 
2. Professional Development (training, technical assistance, and coaching matched to level and type of need) 
e Provide professional development based upon implementation assessment and building needs to reach a 
level of minimal level of proficiency in each of the four essential elements to begin implementation for 
the first year. 
o Collaborative Data Teams 
oO Common Formative Assessment 
0 Data-based Decision-making 
oO Selected Effective Teaching Practices 
e Provide technical assistance and coaching to building to support/monitor implementation the practices 
learned through training to ensure implementation fidelity and adherence to building commitments for 
data reporting/submission. 


Year 2 & 3 Buildings 
1. Refresher and/or re-orientation, implementation assessment, and planning 
e Use the Missouri Collaborative Work Overview Learning Package to orient new staff in the building to 
the CW or provide a refresher. [Estimated time = 45 minutes] 
e Use the Practice Profiles to assess CW building’s level of implementation. 
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oO Assist building in selecting Teaching Practice(s) to focus on for the year 
oO Based upon results of the assessment and selection of effective teaching practice, plan CW 
professional development for the year with appropriate building staff 
2. Professional Development (training, technical assistance, and coaching matched to level and type of need) 
e Provide professional development based upon implementation assessment and building needs to reach 
advanced levels of proficiency in each of the four essential elements to begin implementation for the 
first year. 
o Collaborative Data Teams 
oO Common Formative Assessment 
oO Data-based Decision-making 
oO Selected Effective Teaching Practices 
e Provide technical assistance and coaching to building to support/monitor implementation the practices 
learned through training to ensure implementation fidelity and adherence to building commitments for 
data reporting/submission. 


When a school has participated in trainings/site support without making adequate progress and the school is not 
honoring the expected commitments necessary to enact change and implement the practices with fidelity, the 
school may not be eligible to continue as a CW building. 


Chart 3 shows the Collaborative Work workflow for participating schools as well as the support provided from 
the Regional Professional Development Centers and the DESE. 
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To support schools in selecting, adopting, and full implementation of effective teaching/learning practices, the 
following educator learning packages have been developed. The RPDCs use the learning package content when 
providing coaching and training. 


Effective Teaching and Learning Practices Learning Packages 


Collaborative Data Teams 
Data-Based Decision-Making 
Common Formative Assessment 


Effective Teaching and Learning Practices Overview 


Assessment Capable Learners 
Classroom Discussion 
Feedback 

Reciprocal Teaching 
Student-Teacher Relationship 
Spaced vs. Massed Practice 
Metacognition 

Direct Instruction 
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Using Technology for Classroom Instruction 
School-Based Implementation Coaching 

Teacher Clarity (coming spring 2015) 

Introduction to Visible Learning (coming spring 2015) 
Technology to Support Collaborative Work (coming spring 2015) 
Cooperative Learning (coming spring 2015) 

Leadership for Collaborative Work (coming spring 2015) 


The contents of each learning package adheres to the following framework. This framework is based on a 
synthesis of research on adult learning principles and effective methods for in-service instruction for educators. 


Chart 4: Learning Package Elements 


Learning Package Elements 


Component Purpose Examples of Content 

Preparation Provide opportunity for learners to e Learning objectives 

engage in the content prior to the e Expectations for the training 

formal training e Preparatory reading reflection exercise 
Opening & Provide an overview of the day, e Session at-a-glance 
introductions including reviewing learner e Introductions 

objectives, outcomes, and essential e Essential questions 

questions e Norms 

e Pre-assessment 

Why the topic is Review the basics and relevance to e Implications for student learning 
important student learning e Ways implementation aligns with MO 


Learning Standards 


Overview of the topic 


Provide learner with core concepts, 
terms, and vision for implementation 


e Core concepts 
e Glossary of terms 
e Implementation example 


Unpacking the topic 


Explore the core components and 
implementation steps 


e Detailed description of the core 
components 

e Rationale for components 

e Detailed implementation steps 


Topic in practice 


Topic in action 


Assessment & 
reflection 


Provide opportunity for learners to 
discuss what application in the 
classroom looks like 


Explore ways for the learners to 
incorporate the new knowledge and 
skills into their teaching 


Provide opportunity for the learners 
to reflect on their learning and 
potential implementation challenges 
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e Detailed description of what 
implementation looks like 

e Group discussion on what 
implementation looks like in a variety of 
contexts 

e Measuring fidelity 

e Using data to inform practice 

e Reflection on what implementation 
would look like in their classrooms 

e Discuss and problem-solve potential 
challenges to implementation and 
fidelity drift 

e Post-assessment learner knowledge 

e Reflect on personal teaching context and 
implementation 


Closing & follow-up 


Provide opportunity for learner to 
outline their implementation steps 
and plans for follow-up coaching 


e Template for outlining implementation 
steps in personal teaching contexts and 
follow-up coaching 

e Additional resources for further learning 


Expectations for High Quality Training and Coaching 
Criteria of quality training and coaching, as well as a system of data collection, have been established and will 
continue to be used. See the description of training and technical assistance on page 12 of this plan. 


Theory of Action 
A graphic illustration that shows the rationale of how implementing the coherent set of improvement strategies 
selected will increase the State’s capacity to lead meaningful change in LEAs and achieve improvement in the 
State-identified Measurable Result(s) for Children with Disabilities. 
Chart 5 
Collaboration Accountability 
through... through ... 
Coaching) through... 
Collaboration with DESE/RPDC teams for capacity and fluency of icati = 
statewide leadership content development RPDCS to deliver high | [Ommuncaren led — 
organizations peer review, and vetting Quality professional 
development 
Collaboration on Quarterly collaboration on| | Alignment of professional p = 
leadership level across | | tools and resources for | |development content and| | 42 cnr septrsnnepe Al 
department and with accountability and Process with Missouri vision & 
measuring progress Teacher Standards 
= Clear expectations for 
i to share training, coaching, & 
ideas _ as 
implementation 
Monthly meetings: SPDG 
Providing guidance and | | management team, RPDC 
support for improving directors, State 
data systems Implementation 
Specialists 
‘ Professional development 
one meal focused on Collaborative 
meeting (management 
team plus RPDC leaders 
and consultants 
Then Missouri educators participating in the Collaborative Work will... 
* Select and implement effective teaching/learning practices, collaborative data teams, date 
based decision, and common formative assessment with fidelity; 
* Hold high expectationsfor allstudents, including SWD. 
* Use data to drive instructional decision-making inorderto betterthe diversity of learning 
needs acrossallstudents. 
Then... 
All students with disabilities inthe pilot schools willdemonstrate improved educational results. 
Optional Description 


The State’s Theory of Action articulates that all levels of the process must accept responsibility for the success 
of the levels below them. This clarifies that while tasks can be assigned/reassigned, the responsibility for 
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ensuring the success of the groups or individuals with those assignments is retained. The end result is that all 
levels of the system must be active participants. This concept strongly influences how and by whom data are 
collected, reviewed, discussed, and communicated. It ensures that all levels actively use data to arrive at 
decisions regarding progress and needed modifications. Data transparency built into the system also allows for 
all levels to challenge the accuracy of any data used in the system. The systems approach provides consistency 
of implementation with many opportunities for input and feedback especially at the development level. The 
Theory of Action also shows that while the system is built to focus on a specific set of skills and practices, 
participating schools have a fair amount of flexibility in deciding which effective teaching/learning practices are 
most appropriate to those schools. 
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